Field of the Invention
The present invention relates to techniques for removing anomalies such as spikes from data. Particularly, the present invention is directed to systems, methods and software programs for removal of anomalies from financial data.
Description of Related Art
Price spikes have become an increasingly common and annoying problem for chartists, particularly those that regularly view intra day financial charts. When a spike comes through, for example, on either a high or a low value, it causes the chart to rescale. Depending on the size of the spike, it can render the chart useless from a practical standpoint.
A variety of approaches have been proposed for mitigating spikes in various contexts. For example, U.S. Pat. No. 4,965,800 discloses a digital signal fault detector using low and high voltage thresholds for spike discriminator. U.S. Pat. No. 4,412,287 discloses a similar approach for removing spikes from price data if new market price exceeds the fluctuation limits allowed by the market regulator. Additional techniques of spike removal in technical systems may be accomplished, for example, by delaying of the input signal as described in U.S. Pat. No. 7,239,494 or by generating an additional reference signal by connecting to different parts of the source object as described in U.S. Pat. No. 7,157,924.
However, these approaches are not satisfactory. For example, earlier approaches to spike removal based on comparison with threshold limits of the signal itself such as in U.S. Pat. No. 4,965,800 and in U.S. Pat. No. 4,412,287 are generally only appropriate for working with narrow range signals. Signals of high dynamic range might be discriminated incorrectly and spikes might not be detected. Moreover, approaches based on comparison with reference signals, such as those in U.S. Pat. Nos. 7,239,494 or 7,157,924 are not useful for price time series.
Furthermore, additional problems with spikes in price data are related to the periods when market is closed (e.g. during the night). During these periods of time important economic events may happen causing significant changes in price, which could be mistakenly identified as spikes by known methods.
Still another problem in spike removal from price data is the fact that price behavior is better described by jump-diffusion models (see for example S. G. Kou, “A Jump-Diffusion Model for Option Pricing”, Management Science, Vol. 48, No. 8, August 2002), not by just a diffusion model. The difference between these two models is an additional “jump component” term in the pertinent stochastic differential equation. This “jump component” describes irregular jumps in price, which could be mistakenly identified as spikes by known methods.
Accordingly, there is a continued need in the art for improved techniques for removal of spikes in data. The present invention provides a solution for these problems.